fix(destination-motherduck): serialize JSON array columns to avoid empty STRUCT failure by devin-ai-integration[bot] · Pull Request #82244 · airbytehq/airbyte

devin-ai-integration · 2026-07-16T20:08:09Z

What

Fixes MotherDuck destination sync failures when an array-typed field contains an empty object (e.g. [{}]), which surface as:

ProgrammingError(_duckdb.InvalidInputException) Invalid Input Error: Attempted to convert a STRUCT with no fields to DuckDB which is not supported
[SQL: register($1, $2)]

This is the failure reported for the Shopify orders stream (line_items, refunds, fulfillments, tax_lines, etc. — all type: ["null","array"] with items: object).

Related to https://github.com/airbytehq/oncall/issues/13118:

https://github.com/airbytehq/oncall/issues/13118

How

_serialize_object_columns() (in destination_motherduck/processors/duckdb.py) pre-serializes columns to JSON strings so PyArrow does not infer struct types that DuckDB rejects on empty structs. Previously it only handled columns whose JSON-schema type was "object", skipping array columns. When an array column contained empty objects, PyArrow inferred list<struct<>> (list of fieldless structs), which DuckDB's register()/INSERT rejects.

The gate is now based on the destination SQL column type rather than the raw schema type:

sql_type = SQLTypeConverter().to_sql_type(properties[col_name])
if not isinstance(sql_type, JSON):
    continue
result[col_name] = [orjson.dumps(v).decode() if isinstance(v, (dict, list)) else v for v in values]

In the CDK type mapping (airbyte_cdk/sql/types.py), both object and array map to the SQL JSON type, so both are now serialized; vector_array maps to a native SQL ARRAY and is correctly left untouched. List values (not just dicts) are now serialized, and None/scalar values pass through unchanged.

Declarative-First Evaluation

Not applicable — destination-motherduck is a Python CDK connector (language:python, cdk:python), not a declarative/manifest connector.

Test Coverage

New unit_tests/test_serialize_object_columns.py: parametrized coverage of _serialize_object_columns (object, array-of-objects, array-of-scalars, nulls, scalar passthrough, non-schema Airbyte columns) plus a regression test that builds the PyArrow table from an array-of-empty-objects column and registers it in an in-memory DuckDB — this reproduces the reported error without the fix and passes with it.
Extended integration_tests/integration_test.py test_write with an array_of_objects_key field containing [{}] / [{"a": 1}, {}], exercising the full write path end-to-end (runs against a local DuckDB file in CI).

Both suites pass locally (poetry run pytest unit_tests/ → 53 passed; test_write[local_file_config] and test_write_dupe[local_file_config] pass).

Review guide

airbyte-integrations/connectors/destination-motherduck/destination_motherduck/processors/duckdb.py — the fix
airbyte-integrations/connectors/destination-motherduck/unit_tests/test_serialize_object_columns.py — new tests
airbyte-integrations/connectors/destination-motherduck/integration_tests/integration_test.py — extended coverage

User Impact

Syncs to MotherDuck with array fields that contain empty objects (e.g. Shopify orders) will no longer fail in the destination write stage. Non-breaking bug fix; patch bump 0.2.4 → 0.2.5-rc.1 (progressive rollout is enabled on this connector, so the -rc.1 suffix is required).

Can this PR be safely reverted and rolled back?

YES 💚
NO ❌

Link to Devin session: https://app.devin.ai/sessions/b5a2e996760a4d5f9157915b4fe0d2cb

Important

Active progressive rollout warning for destination-motherduck.

(Click to Approve:) Bypass the active progressive rollout warning for destination-motherduck in the PR comment here.

…pty STRUCT failure Co-Authored-By: bot_apk <apk@cognition.ai>

devin-ai-integration · 2026-07-16T20:08:11Z

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

Disable automatic comment, CI, and merge conflict monitoring

Co-Authored-By: bot_apk <apk@cognition.ai>

github-actions · 2026-07-16T20:08:38Z

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

PR Slash Commands

Airbyte Maintainers (that's you!) can execute the following slash commands on your PR:

🛠️ Quick Fixes
- /format-fix - Fixes most formatting issues.
- /bump-version - Bumps connector versions, scraping changelog description from the PR title.
  - Bump types: patch (default), minor, major, major_rc, rc, promote.
  - The rc type is a smart default: applies minor_rc if stable, or bumps the RC number if already RC.
  - The promote type strips the RC suffix to finalize a release.
  - Example: /bump-version type=rc or /bump-version type=minor
- /bump-progressive-rollout-version - Alias for /bump-version type=rc. Bumps with an RC suffix and enables progressive rollout.
❇️ AI Testing and Review (internal link: AI-SDLC Docs):
- /ai-prove-fix - Runs prerelease readiness checks, including testing against customer connections.
- /ai-canary-prerelease - Rolls out prerelease to 5-10 connections for canary testing.
- /ai-review - AI-powered PR review for connector safety and quality gates.
📝 AI Documentation:
- /ai-docs-review - AI-powered documentation review for PRs with connector changes.
- /ai-create-docs-pr - Creates a documentation PR for connector changes, stacked on the current PR.
🚀 Connector Releases:
- /publish-connectors-prerelease - Publishes pre-release connector builds (tagged as {version}-preview.{git-sha}) for all modified connectors in the PR.
- /enable-autopilot-rollouts - Enables autopilot progressive rollouts for the modified connector(s) in the PR, remediating "autopilot rollouts not enabled for {connector-name}" auto-merge blockers. Sets defaultRolloutMode: autopilot and enableProgressiveRollout: true, preserving any existing autopilotConfig.
  - Optional args: connector=<CONNECTOR_NAME> (defaults to the modified connectors in the PR), strategy=fast|slow|default (defaults to fast).
  - Example: /enable-autopilot-rollouts or /enable-autopilot-rollouts connector=source-faker strategy=slow
☕️ JVM connectors:
- /update-connector-cdk-version connector=<CONNECTOR_NAME> - Updates the specified connector to the latest CDK version.
  Example: /update-connector-cdk-version connector=destination-bigquery
🐍 Python connectors:
- /poe connector source-example lock - Run the Poe lock task on the source-example connector, committing the results back to the branch.
- /poe source example lock - Alias for /poe connector source-example lock.
- /poe source example use-cdk-branch my/branch - Pin the source-example CDK reference to the branch name specified.
- /poe source example use-cdk-latest - Update the source-example CDK dependency to the latest available version.
⚙️ Admin commands:
- /force-merge reason="<REASON>" - Force merges the PR using admin privileges, bypassing CI checks. Requires a reason.
  Example: /force-merge reason="CI is flaky, tests pass locally"

📚 Show Repo Guidance

Helpful Resources

Breaking Changes Guide - Breaking changes, migration guides, and upgrade deadlines
Developing Connectors Locally
Managing Connector Secrets
On-Demand Regression Tests
#connector-ci-issues
#connector-publish-updates
#connector-build-statuses

📝 Edit this welcome message.

github-actions · 2026-07-16T20:10:20Z

Detected `destination-motherduck` Active Rollout: `true`

Important

Active progressive rollout warning for destination-motherduck.

To bypass this warning, click on the matching checkbox in the PR description. Look for the checkbox text:

(Click to Approve:) Bypass the active progressive rollout warning for destination-motherduck in the PR comment

Rollout version: 0.2.4
Rollout state: in_progress
Rollout last updated by: Airbyte Support Bot <airbyte-support-bot@airbyte.io>
Open Connector Rollout Manager in Retool to clean up or close out this rollout if appropriate.

Version on `master` Branch: `0.2.4`

RC marker on master branch: false

PR Description Checkbox Status

Bypass checkbox checked: false

ℹ️ More Information

Show/hide details...

🤔 What happens if this PR is merged

Checking the checkbox will allow the PR to merge, but it does not necessarily stop the active rollout by itself. The result of the PR merging depends on what connector version is published.

Expected outcomes by type of version number change:

If connector version is not modified in this PR...

No new connector version should be released, and the active rollout should continue unchanged.

If the connector version increments to a higher `-rc` version...

After this PR is merged, the new RC will be published and registered, replacing the active RC marker. When the new RC is registered, the platform cancels any existing non-terminal rollout for this connector without unpinning actors.

After merging, you still need to start the new rollout. During start, pinned actors from the previous rollout can be moved to the new RC.

If the connector version changes from RC to non-RC (GA) version...

You should not merge the PR unless/until the RC has been finalized as canceled. See above Rollout state for detected status.

[!Warning]
This PR should not be merged if the RC rollout is still active. First finalize the active rollout as successful or cancel it in Connector Rollout Manager.

When you finalize an RC rollout as successful, the platform triggers a promotion workflow that strips the -rc suffix, removes stable-version registryOverrides, disables progressive rollout, force-merges that promotion, and unpins actors.

🔁 How to rerun this check

To rerun the check, simply check and uncheck the box, or else modify the PR description and/or title in any way.

Alternatively, you can find the Active Progressive Rollout CI workflow and manually rerun it (although this is generally slower than the above methods).

This comment will be updated as PR and/or rollout status changes.

Workflow run

github-actions · 2026-07-16T20:12:16Z

`destination-motherduck` Connector Test Results

58 tests 56 ✅ 20s ⏱️
2 suites 2 💤
2 files 0 ❌

Results for commit 03b602d.

♻️ This comment has been updated with latest results.

github-actions · 2026-07-16T20:13:37Z

Deploy preview for airbyte-docs ready!

Project:	`airbyte-docs`
Status:	✅ Deploy successful!
Preview URL:	https://airbyte-docs-o01nan3hb-airbyte-growth.vercel.app
Latest Commit:	`03b602d`

Deployed with vercel-action

fix(destination-motherduck): serialize JSON array columns to avoid em…

7847b06

…pty STRUCT failure Co-Authored-By: bot_apk <apk@cognition.ai>

docs(destination-motherduck): correct changelog PR link

03b602d

Co-Authored-By: bot_apk <apk@cognition.ai>

Octavia Squidington III (octavia-squidington-iii) added the connectors/destination/motherduck label Jul 16, 2026

devin-ai-integration Bot added the hyd-fix Hydra: ai-fix stage has run label Jul 16, 2026

devin-ai-integration Bot assigned Airbyte Support Bot (airbyte-support-bot) Jul 16, 2026

devin-ai-integration Bot mentioned this pull request Jul 16, 2026

ci(hydra): Fail ineligible PRs before scope analysis #82106

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(destination-motherduck): serialize JSON array columns to avoid empty STRUCT failure#82244

fix(destination-motherduck): serialize JSON array columns to avoid empty STRUCT failure#82244
devin-ai-integration[bot] wants to merge 2 commits into
masterfrom
devin/1784232451-motherduck-empty-struct-arrays

devin-ai-integration Bot commented Jul 16, 2026 •

edited by github-actions Bot

Loading

Uh oh!

devin-ai-integration Bot commented Jul 16, 2026

Uh oh!

github-actions Bot commented Jul 16, 2026

PR Slash Commands

Helpful Resources

Uh oh!

github-actions Bot commented Jul 16, 2026 •

edited

Loading

🤔 What happens if this PR is merged

🔁 How to rerun this check

Uh oh!

github-actions Bot commented Jul 16, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 16, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

devin-ai-integration Bot commented Jul 16, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

How

Declarative-First Evaluation

Test Coverage

Review guide

User Impact

Can this PR be safely reverted and rolled back?

Uh oh!

devin-ai-integration Bot commented Jul 16, 2026

🤖 Devin AI Engineer

Uh oh!

github-actions Bot commented Jul 16, 2026

👋 Greetings, Airbyte Team Member!

PR Slash Commands

Helpful Resources

Uh oh!

github-actions Bot commented Jul 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Detected destination-motherduck Active Rollout: true

Version on master Branch: 0.2.4

PR Description Checkbox Status

ℹ️ More Information

🤔 What happens if this PR is merged

🔁 How to rerun this check

Uh oh!

github-actions Bot commented Jul 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

destination-motherduck Connector Test Results

Uh oh!

github-actions Bot commented Jul 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

devin-ai-integration Bot commented Jul 16, 2026 •

edited by github-actions Bot

Loading

github-actions Bot commented Jul 16, 2026 •

edited

Loading

Detected `destination-motherduck` Active Rollout: `true`

Version on `master` Branch: `0.2.4`

github-actions Bot commented Jul 16, 2026 •

edited

Loading

`destination-motherduck` Connector Test Results

github-actions Bot commented Jul 16, 2026 •

edited

Loading